Test Set

This notebook is used as a "first look" of the kaggle competition "Google Landmark Recognition 2019" test set.

Because the available test set of the 2020 comeptition is witouth its ground truth table we will use 2019 competition test set to evaluate our algorithm.

The 2019 competition test set was released as CSV file with three columns: id (16-character string), landmarks (space-separated list of integer landmark IDs, or empty if no landmark from the dataset is depicted) and Usage (either "Public" or "Private", referring to which subset the image belongs to).

It can be found here.

As we can see there are a lot of NaN values in the landmarks column. That mean that a lot of the objects in the test set are not landmarks but non-landmarks.

We would like to look at the images from the train set:

As we saw from the dataframe, the train set contain a lot of out of domain images, i.e. image that are not landmark at all but images with no clear subject that can be protrait, group of people, animals, plants, food, etc.

We will look now only on the landmarks inside the test set.

As we can see, most of the objects in the test set are non-landmarks.

We would like to look at the images from the test set that identified as landmarks:

As we can see, all the images above shows a variety of landmarks

We'll now check few of the test set properties:

As we can see, the test set composed out of 117,577 images that divided into 849 classes. Only 1972 pictures out of the 117,577 images in the test set are actually landmarkd, e.i. only 1.67 %

We would like to look at the different classes in the test set, we we'll start with the top classes:

As we can see from the top result, some images can relate to more than one class. We would like to know how many images can realte to more than one

As we can see, one image can relate to different classes (up to 4).

As we can see from the above, when we first split and then concatenate the landmarks coulmn, it changed the amount of recurrence of some classes. It also changed the amount of classes to 852 classes.

We would like to plot the classes histogram:

As we can see from the histogram, the classes are scattered across the x axis.

We would like to inspect the top and bottom classes more carefulley:

We will display some images from some of the top classes:

As we can see from the graphs above, the top classes don't appear very often in the test set and a lot of classes appear only once in the train set.

Test set images vs train set images

We would like to see how some images from the test set look like in the train set:

As we can see from the images above, somtimes the train and the test set images are not similar. They can differ in color, angel, brightness, lighting.

In class 6798 for example, the landmark is theatre des champs elyses. The train set images include images of the theater from the inside and from the outside while the test images include only images from the inside.

In class 86283, the landmark is luxembourg gardens. The train set images are images taken from all around the graden which is quite big while it look like the test images are from one part of the park.

In class 70439, the landmark is chancery lane. The train image include images from different part of the lane while the test image is a view of all the lane.

Those differences between the test set and the train set will make our recognition mission very difficult and non-trivial.

This is the 2019 test set we will use to evaluate our algorithm.